Goodness-of-fit and confidence intervals of approximate models

To test whether the model fits the data well, a goodness-of-fit (GOF) test can be used. The chi-square GOF test is often used to test the null hypothesis that a function describes the mean of the data well. The null hypothesis with this test is rejected too often, however, because the nominal significance level (usually 0.05) is exceeded. Alternatively, the level of Hotelling’s test is accurate if a fixed hypothesis for the mean is available. In many situations, however, only an estimate of the mean is available, and so the level of Hotelling’s test may also be incorrect. An approximate version of Hotelling’s test is suggested as a GOF test. It is shown that this requires only an adjustment of the degrees of freedom of Hotelling’s original test. GOF tests assume that the model is either correct or incorrect whereas in model specification it is often assumed that the model is an approximation. Consequently, for approximate models a GOF test will mostly indicate that the model does not fit. It is therefore suggested that a measure of approximation to the true model could be used to get an indication of how bad the approximate model is. It is also shown that correct confidence intervals can be obtained from when using an approximate model. The results are applied to data from the daily news memory test.